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I  ABSTRACT 


It  is  commonly  supposed  that  L  ■  XW  applies  to  "almost  any" 


queueing  system,  with  X  the  mean  customer  entrance  rate,  L  the 

asymptotic  expectation  of  the  number  of  customers  in  the  system, 

* 

and  W  the  asymptotic  sojourn  time  expectation.  We  study  the 

formula  for  irreducible  positive  recurrent  Markov  queueing  systems 

whose  state  vector  Z  consists  of  entries  representing  queue  lengths 

at  the  respective  service  stations ;  such  a  model  permits  blocking, 

finite  capacities,  jockeying,  state -dependent  or  random  routing, 

bulk  and/or  Erlang  service,  and  variable  arrival  and  service  rates. 

To  define  waiting  times  under  various  queueing  disciplines,  Z  is 

augmented  by  a  customer  location  process  to  yield  the  new  Markov 

process  Y  »  (Z,U).  It  is  shown  that  the  standard  regenerative 

process  proof  of  Little's  equality  fails  in  the  absence  of  further 

hypotheses;  however,  additional  assumptions  assure  the  validity  of 

L  *XW  for  a  broad  variety  of  queueing  disciplines.  Generalizations 

of  the  formula  are  obtained  for  (a)  state  dependent  customer 

arrival  rates,  (b)  periodicity  of  the  number  of  customers  per  busy 

period,  and  (c)  multiple  classes  of  customers.  Non-Markovian 

# 

queueing  systems  are  briefly  discussed. 


Key  Words:  Little's  formula,  queueing  networks,  sojourn  times, 
transit  times,  Markov  jump  processes,  regenerative 
processes 
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0.  INTRODUCTION 


According  to  a  recent  survey,  the  "...  average  waiting  and 
response  times  can  be  obtained  easily  from  Little’s  formula  L  »  XW 
..."  fdr  customers  in  Markovian  queueing  systems  of  general  type 
[14] .  This  would  seem  to  be  supported  by  the  classical  queueing 
literature  [12]  [18] ,  in  which  the  reference  is  to  queueing  systems 
rather  than  one  server.  Although  these  assessments  may  be  correct 
in  principle,  a  closer  examination  of  the  applicability  of  the 
formula  discloses  some  subtleties,  which  do  not  occur  in  a  G/G/l 
queue,  but  may  well  be  encountered  in  a  complex  queueing  network. 
These  considerations  affect  not  only  the  form,  but  even  the 
validity  of  the  relation  L  *  XW.  Among  the  deviations  from 
G/G/l  behavior  that  preclude  a  straightforward  argument  are 
variable  entry  rates,  batch  inputs,  periodicity  in  the  number  of 
customers  in  a  busy  cycle,  and  some  (likely  pathological)  non¬ 
existent  expectations. 

Little's  formula  is  amenable  to  diverse  interpretations 
(e.g.,  as  a  relation  for  time  averages  [18]),  but  we  shall  confine 
ourselves  to  asymptotic  expectations.  If  L(t),  the  total  number 
in  the  system  at  time  t,  converges  in  distribution  to  L,,,,  we  call 
L  «  ECL,,) .  Analogously,  when  Wn,  the  system  sojourn  time  of  the 
n'th  customer,  converges  in  distribution  to  W,,,  we  let  W  »  ECW^). 
Finally,  X  is  the  average  rate  of  customer  entry  into  the  system. 

In  common  with  many  other  studies  on  queueing  networks  [1] 

[2]  [3]  [11]  [13]  [16]  ,  we  shall  represent  the  state  of  the  network 
by  a  Markov  jump  process  Z  for  positive  time.  Roughly  speaking, 
Z(t)  is  a  complete  descriptor  of  the  location  and  type  of  customer 
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within  the  network  at  time  t.  As  the  references  just  listed 
indicate,  the  Markov  representation  is  adequate  to  model  a  variety 
of  computer-communication  systmes,  including  different  queueing 
disciplines,  random  routings,  blocking,  state-dependent  entry 
and  service  rates  (e.g.,  finite  buffers  and  multiple  servers, 
respectively),  non-exponential  service  time  distributions, 
different  classes  of  customers,  and  the  like.  The  reader  is 
referred  to  the  above  references  for  applications  and  examples. 

That  the  relation  L  *  AW  may  fail  even  for  a  simple  Markov 
system  is  indicated  by 

Example  0.1:  The  system  consists  of  two  exponential  tandem- 
connected  servers,  with  a  Poisson  input  of  intensity  X  to  the 
first  server.  The  state  is  then  described  by  a  tuple  z  *  (z^,z2) . 
Now  suppose  blocking  and  finite  capacity  permit  only  certain 
direct  transitions,  viz.  (0,0)-Kl,0)-+-(2,0)-*-(l,l)-*-(0,2)-*CO,l)-*-(0,0) . 
Since  the  WR  have  distinct  expectations  according  as  n  is  odd  or 
even,  the  sequence  Wn  cannot  converge  in  distribution,  nor  is  there 
even  any  meaning  to  a  limit  of  E(Wn).  Further,  the  arrival  rate 
to  the  system  may  well  be  constant  at  x,  but  this  does  not  reflect 
the  actual  entry  rate,  which  is  some  variable  quantity.  Conse¬ 
quently  L  *  XW  makes  no  sense,  nor  is  it  obvious  that  there  is 
a  "fixed  up"  version  of  this  formula. 

This  example  illustrates  the  folly  of  an  automatic  assumption 
that  L  ■  XW  must  hold  for  a  stable  Markov  system,  Apparently, 
variable  entry  rates  and  busy  cycle  periodicity  play  some  role 
in  the  validity  of  the  formula.  We  shall  also  see  that  there  are 
other  hypotheses  which  we  cannot  omit. 


.  ...  L—t 
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We  shall  begin  by  summarizing  standard  arguments  leading  to 
Little's  formula.  Next,  we  examine  Markov  queueing  networks  to 
determine  their  conformance  to  the  hypotheses  underlying  a  proof 
of  Little's  formula.  Some  of  these  hypotheses  are  automatically 
met  by  any  queueing  system  represented  by  a  well-behaved  Markov 
process,  whereas  others  require  separate  assumptions,  or  lead  to 
a  modification  of  L  ■  AW.  Among  the  extensions  are  variable  input 
rates  and  multiple  customer  classes.  Finally,  we  point  out  that 
certain  of  the  results  are  applicable  to  non-Markov  networks. 

I .  BACKGROUND 

A  Markov  queueing  network  will  be  represented  by  a  vector 
Markov  jump  process  defined  on  a  state  space  whose  elements 
z  -  (z^ ,z2 > . . . ,zr)  have  non-negative  integer-valued  components 
z^  that  may  be  thought  of  as  the  respective  number  of  customers 
located  at  the  r  service  stations;  this  is  consistent  with  many 
literature  descriptions  of  computer-communication  system  models 
[1]  [3] [9]  [14] .  Accordingly,  we  shall  begin  with  a  regular  right- 
continuous  Markov  jump  process  Z  ■  (Z^^*. . .  ,Zr)  having  this 
state  space,  and  defined  for  times  t  ^  0.  The  state  0  »  (0,0,..., 0), 
corresponding  to  an  empty  system,  is  supposed  positive  recurrent. 
Since  only  asymptotic  properties  are  of  interest,  we  shall  take 
Z(0)  »  9  without  loss  of  generality.  Central  to  the  upcoming 
derivations  are  the  stopping  (hitting)  times  C Cn } ;  CQ  »  0,  and 
Cn  is  the  time  of  the  n'th  entrance  of  the  system  into  the  state 
0.  At  this  juncture,  we  only  note  that  (Cn>  constitutes  a 
renewal  process  with  finite  inter-renewal  periods  satisfying 


E(Cn+i*Cn)  *  iTq'1  >  0  (cf.  [5],  Section  8.5  and  Prop.  10.1.12), 
in  which  ttq  denotes  the  invariant  probability  measure 
lim  P[Z(t)*0]  -  7rQ. 

For  the  present,  let  us  understand  the  sojourn  times  CvO 
and  number  of  customers  in  the  system  {L(t) >  in  intuitive  terms; 
in  any  case,  {Wn>  cannot  be  defined  entirely  in  terms  of  Z, 
since  individual  sojourn  times  are  also  heavily  dependent  on  the 
queueing  discipline  (not  reflected  in  Z) .  For  now,  we  review 
a  derivation  of  Little's  formula  for  a  later  analysis  of  its 
application  to  a  queueing  network  specified  by  the  Markov  process 
Z.  Of  the  modem  proofs  of  Little's  equality,  perhaps  the  best 
known  and  most  general  are  those  of  Jewell  [12]  and  Stidham  [18] 
[19].  Stidham  points  out  that  the  relation,  in  terms  of  time 
averages,  is  actually  non- stochastic ;  under  further  assumptions, 

L  *  XW  can  be  retrieved  for  a  random  system.  Jewell's  approach 
(see  also  [6],  Section  1.6ii)  is  more  direct,  attaining  its 
results  through  use  of  regenerative  process  theory  and  Wald's 
lemma.  Unfortunately,  Jewell  ignores  the  implications  of  period¬ 
icity  and  non-uniform  arrival  rates.  It  is  easy  to  state  the 
propositions  underlying  Jewell's  argument,  and  we  do  so  for 
future  reference  without  proof. 

Theorem  1 . 1  (cf.  [18],  Theorem  I;  [12],  Lemma  1):  For  a  busy 
cycle  over  the  interval  [0,C]*,  and  entailing  N  customers 

/C  N 

L(t)dt  “  X  Wn  »  Cl.l) 

n-1 

where  W  is  the  total  system  sojourn  time  for  the  n'th  customer 
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to  enter  the  system,  and  L(t)  *  ||  Z(t)||  (i.e.,  the  ^  norm  of  Z(t)). 
Theorem  1.2  ([6],  Theorem  S.l):  Let  Z  be  a  regenerative  process 
([6],  Definition  2.1)  with  respect  to  the  renewal  sequence  {Cn>, 
where  the  renewal  intervals  are  non-arithmetic  with  finite  expecta¬ 
tion.  If  the  expectation  on  the  right  of  (1.2)  exists,  L(t) 
converges  in  distribution  to  a  random  variable  L^  ,  and 

E(LJ  *  {E (C)  }”1E{|*C L(t)dtJ  .  (1.2) 

Remarks:  Without  loss  of  generality,  we  have  taken  CQ  *  0,  and 
used  C  for  ^  in  (1.2).  Although  it  does  not  affect  the  statement 
of  the  Theorem,  we  prefer  to  let  the  Cn  correspond  to  times  the 
system  becomes  empty,  rather  than- -as  convention  would  have  it-- 
those  instants  the  empty  system  receives  a  first  customer.  Our 
definition  assures  that  Z  is  in  the  specific  state  Z(Cn)  »  0,  so 
that  we  may  use  the  strong  Markov  property. 

Theorem  1.3  ([6],  Theorem  3.2):  Let  {Wn>  be  a  regenerative  process 

with  respect  to  the  renewal  sequence  (N^L  were  is  the  total 

number  of  customers  entering  the  system  over  the  interval  (0,0^] . 

If  the  number  of  customers  in  a  busy  period  has  finite  expectation 

and  is  aperiodic,  the  Wn  converge  in  distribution  to  a  random 

variable  W  .  and 
00 


N 

EOT.)  -  {E(N)}'1E(2^Wn}  .  (1.3) 

n*l 

Remark:  As  in  Theorem  1.1,  we  take  CL  *  0  and  write  N  for  N, . 

o  l 
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Theorem  1.4  (Wald’s  Lemma):  Assume  {A^}  is  a  sequence  of  i.i.d. 
random  variables  of  finite  mean.  Let  N  be  a  random  variable  such 
that  the  event  (N  <_  n}  is  stochastically  independent  of  A^  for 
all  k  >_  n+1.  Then  if  C  is  defined  as 

N 

C  -  2X  Cl. 4) 

n=l 

we  have 

E  (C)  *  E  (A)  E  (N)  .  (1.5) 

Remark:  If  the  busy  cycle  is  conventionally  defined  to  begin 
with  the  arrival  of  a  customer  into  the  empty  system,  the  A^  are 
interarrival  times,  and  N  is  as  in  Theorem  1.3,  then  (1.4)  correctly 
describes  the  busy  cycle  (cf.  [6],  p.l);  hence  (1.5)  holds  for 
i.i.d.  interarrival  periods  at  the  constant  rate  \  -  [E(A)]"1. 
However,  (1.4)  is  not  appropriate  for  our  busy  cycles  (Cn,Cn+1]  > 
which  are  assumed  to  begin  when  the  system  first  becomes  empty. 

If  the  conditions  of  the  preceding  theorems  are  satisfied, 
we  may  combine  equations  (1.1),  (1.2),  (1.3)  and  (1.5)  to  yield 
Little's  formula  with  L  «  EfL,,,)  and  W  «  EfW^).  As  is  well  known, 
the  formula  is  applicable  to  the  stable  G/G/l  queue --provided 
we  ignore  periodicities.  For  instance,  the  special  case  of  the 
D/G/l  queue  (e.g.,  An  «  1)  implies  a  periodic  L(t)  that  does  not 
converge  in  distribution.  We  also  find  that  an  adjustment  must 
be  made  for  fixed  bulk  arrivals,  and  that  Theorem  1.4  fails 
altogether  if  the  G/G/l  queue  is  modified  by  introducing  state 
dependence  in  the  arrival  process.  Although  arrival  periodicity 
is  inconsistent  with  the  Markov  property  as  it  appears  in  a 


Markov  queueing  network,  we  cannot  exclude  stochastic  dependence 
of  the  state,  the  service  process,  and  the  arrival  process.  These 
complex  behavior  modes  suggest  that  Little's  relation  may  hot  be 
generally  applicable,  as  Example  0.1  has  already  indicated.  We 
are  therefore  led  to  a  careful  construction  and  examination  of 
a  Markov  network  queueing  model,  for  which  we  ascertain  the  extent 
to  which  the  hypotheses  of  the  above  theorems  are  satisfied.  When 
these  theorems  are  not  directly  applicable,  we  inquire  whether 
the  Markov  model  must  meet  additional  restrictions,  or  whether 
the  original  model  can  lead  to  modified  forms  of  Theorem  1.1  to 
1.4. 

2.  APPLICABLE  PROPERTIES  OF  MARKOV  QUEUEING  NETWORKS 

Because  the  queueing  network  state  Z  is  a  Markov  process, 
the  hypotheses  of  Theorems  1.1  to  1.4  are  met  in  varying  degrees. 

A  portion  of  this  section  is  devoted  to  reviewing  these  assumptions 
in  light  of  the  Markov  character  of  Z,  and  determining  whether 
additonal  suppositions  are  needed  for  a  proof  of  Little's  formula. 
We  see,  however,  that  Z  lacks  altogether  the  customer  identifica¬ 
tion  essential  to  tracing  a  customer's  progress  through  the 
system  (for  any  queueing  discipline)  ,  and  relating  his  emergence 
from  the  system  to  his  entry;  hence,  it  is  impossible  to  define 
the  sojourn  times  Wn  without  further  structure.  Augmentation  of 
Z  to  a  new  Markov  jump  process  will  prove  useful  in  this  regard, 
eventually  leading  to  forms  of  Little's  equation  that  are  robust 
respective  to  queueing  disciplines. 

We  recall  that  Z  is  a  regular  right-continuous  Markov  jump 
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process  for  which  9  is  a  positive  recurrent  state;  to  avoid 
trivialities,  we  require  that  0  not  be  absorbing.  We  can  now 
characterize  the  Markov  process  Z  completely  in  terms  of  the  times 
of  jump  T  ,  and  the  values  of  Z  on  its  intervals  of  constancy. 

If  {Zn}  is  defined  by 

Zn  *  Z(Tn)  ,  (2.1) 

{Zn>  and  the  tuple  {(Zn>Tn))  are  both  Markov  processes,  as  is 
evidenced  by  (see  [S],  Chapter  8,  Section  3) 

P[Zn+1»z,Tn+1-Tn>t|Zn*x,A]  =  qx2exp[-ct(x)t]  .  (2.2) 

Here  A  is  any  event  in  the  sigma  algebra  generated  by  (Z°,...,Zn, 

T  ,...,T  }  ,  and  0  <  a(x)  <  «.  The  q _  are  transition  matrix 

o  n  xz 

elements  for  the  discrete  Markov  process  {Zn}.  Further,  (2.2) 
is  related  to  the  infinitesmal  operator  matrix  for  Z  in  the 
sense  that  the  matrix  entries  (sometimes  interpreted  as  flow 
intensities)  are  a  3  a(x)q  unless  x  =  z,  in  which  case 

axx  ■  -'W' 

Because  we  demand  that  Z  be  a  queueing  process,  we  restrict 

possible  positive  q  to  those  consistent  with  queueing  models. 

X  z 

Thus,  the  transition  probability  q  a  0  except  (perhaps)  in  the 
following  three  situations: 

k  customers  enter  the  system  at  service  station  i,  which  means 

z  -  x+ke.^  ;  (2.3) 

(e^  is  the  unit  vector  whose  i’th  component  is  unity);  k  customers 
depart  from  station  i  and  exit  the  system,  i.e., 

(2.4) 


z  *  x-ke.^ 
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or  k  customers  transfer  from  service  station  i  to  arrive  immediately 


at  j ,  viz. 


z  ■  x-ke^+ke^ 


(2.5) 


While  the  q^  and  (Zn>  specify  queue  lengths  and  their  variation, 

they  fail  to  reveal  the  location,  routing  and  departure  of 

individual  customers.  To  speak  of  sojourn  times,  therefore,  we 

must  introduce  an  auxiliary  process,  the  queueing  discipline 

process  {Un}.  For  the  empty  system  (i.e.,  Zn  *  9) ,  Un  *  8. 

Within  each  busy  cycle,  Un  includes  for  each  customer  a  triple 

(m,q  .  jm) »  in  which  m  is  the  customer’s  order  of  arrival  in 
^m  -’m 

chronological  order,  qm  (with  0  <  qm  <  r)  the  queueing  station 

to  which  he  is  assigned,  and  jm  (with  0  <_  j  <  z£  )  his  position 

Hm 

at  station  q  .  The  entries  q„  “  j  *  0  are  reserved  for  customers 
m  m  m 

m  who  have  already  left  the  system.  Of  course,  Un  is  required 
to  be  consistent  with  Zn.  Thus,  a  transition  (2.3)  introduces 
new  triplets  to  Un,  while  a  departure  (2.4)  reduces  k  appropriate 
pairs  qm  and  jffl  to  zero.  Along  with  any  type  of  transition, 
there  may  also  be  a  permutation  of  customer  positions. 

It  is  essential  that  the  queueing  discipline  process  be  of 


the  functional  form 


^(Un,Zn,Zn+1,  5n+1) 


(2.6) 


where  the  £n  are  i.i.d.  variates.  A  great  variety  of  queueing 
disciplines  can  be  represented  in  this  fashion.  These  include 
FCFS,  LCFS,  random  service  order,  and  even  state-dependent 
disciplines.  However,  we  cannot  admit  discipline  dependencies 
that  reach  across  different  busy  cycles,  or  disciplines  that 
depend  on  states  prior  to  those  indicated  by  (2.6). 
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By  virtue  of  (2.6),  {(Zn,Un)j  is  a  Markov  chain,  which  in 

I 

turn  generates  to  continuous  parameter  Markov  process  Y,  where  i 


Y (t)  -  (Z^)  for  Tn  <  t  <  Tn+1  ;  (2.7) 

see  again  [5],  Section  8.3  for  details.  We  note  for  future 
reference  that  if  G^  is  the  hitting  time  at  which  Z(Ck)  becomes 
null  for  the  k'th  time  assumes  the  same  role  for  Y.  The  new 
process  Y  is  used  to  define  sojourn  times.  First,  let  1^  m(t)  *  1 
if  <  t  <  Ck+i  and  if  Y  includes  the  m*th  customer  through  the 
triple  Cm , qm » j m)  with  nonzero  qm  and  jm;  otherwise,  1^  m(t)  *  0. 
With  the  aid  of  this  indicator  function  we  can  define  unambiguously 
the  customer  sojourn  times  by 


W  ,  *T 
m+Nk 


(t)dt 


(2.8) 


Here  is  the  total  number  of  customers  in  the  first  k  busy 
cycles.  More  precisely,  the  number  of  arrivals  in  (0,t]  is 


M(t) 


T 


Ul  Z(T±)  ||  -||  Z(T.-)||  ]  + 


(2.9) 


and  »  M(Ck) .  This  definition  of  is  consistent  in  the  sense 
that  (Nk  -  Nk  l)  arrivals  are  indicated  by  U  over  C C ^ ^ , C^] . 

Let  us  now  turn  to  the  contents  of  Theorem  1.1.  We  state 
and  prove 

Theorem  2.1:  If  (Un) is  consistent  with  (Zn),  the  assertion  (1.1) 
of  Theorem  1.1  is  valid  for  the  queueing  system  Y. 

Proof:  At  t  ■  0,  we  have  L (0 )  ■  0  as  well  as  IQ  m(0)  ■  0  for  all  n. 
For  each  jump  in  L(t)  ■  ||  Z(t)  ||  ,  there  is  a  corresponding  jump 
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in  the  sum  of  the  In  ft) ,  that  is 

u  ,m 

QO 

L(t)  ■  E  ‘o,.*'!  •  (2-10> 

m*l 

On  the  right  side  of  (2.10),  the  terms  are  identically  zero  for 
m  >  N^,  so  that  the  sum  is  adjusted  accordingly.  If  we  now 
integrate  both  sides  of  (2.10)  on  (O.C^] ,  we  obtain  the  desired 
result  from  (2.8). 

Remark:  We  again  note  that  (1.1)  is  "discipline  robust"  with 
respect  to  any  queueing  discipline  obeying  the  consistency 
condition. 

We  have  seen  in  Section  I  that  regenerative  processes  are 
central  to  Jewell's  proof  of  Little's  formula.  We  now  connect 
Markov  queueing  networks,  as  described  by  Y,  with  regenerative 
process  theory.  Most  useful  for  this  purpose  is  the  strong 
Markov  property,  whose  relevant  parts  we  present  below;  a  proof 
can  be  found  in  [4],  Section  II. 9,  Theorem  5.  We  must  again 
begin  with  certain  definitions  that  make  precise  the  "past"  and 
"future"  with  respect  to  the  Cn-  To  begin  with,  let ^  * 
o{Y(s),s<t),  that  is,  ^  is  the  o-algebra  generated  by  Y  up  to 
time  t.  Next,  the  pre-T  o-algebra is  specified  in  standard 
fashion  for  a  stopping  time  T  as  follows:  if  An{T<t}e$Ft  for 

each  t.  We  also  define  a  post-T  ("future")  o-algebra  Let 

_  rr 

ST  be  the  translation  operator  (SjX) (t)  ■  X(t+T) ;  then 9  * 

o((StY) (t) ,t>0}.  With  this  notation,  we  state 
Lemma  2.2:  Let  0  ■  Cq  <  <  C2  <. . .  be  a  sequence  of  stopping 

times  such  that  P[Y(Cjc)  ■  0]  ■  1  for  k  »  0,1,2,...  If  AeJ£  and 
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then 


PCAnr)  -  PCA)P(r) 


(2.11) 


Moreover,  probabilities  are  invariant  under  translations  Sr  ; 

Lk 

P[(sc  (Y(t1),Y(t2),..,Y(tm)))eB]  -  P[(Y(tx)  ,Y(t2)  , . .  ,Y(ta))eB] .  (2.12) 

k 

Remark:  Lemma  2.2  would  fail  to  hold  if  we  had  taken  the  as 
arrival  times  of  the  first  customer  into  the  empty  system,  unless 
qQz  >  0  for  exactly  one  z,  implying  that  the  first  arrival  number 
and  location  must  be  fixed.  To  avoid  such  a  restriction  we  have 
defined  the  as  already  indicated,  contrary  to  the  usual  con¬ 
vention  of  the  initiation  of  the  busy  cycle.  With  the  aid  of 
Lemma  2.2,  it  is  easy  to  prove 

Theorem  2.3:  Theorem  1.2  holds  for  the  queueing  system  Y. 

Proof:  It  was  noted  near  the  beginning  of  Section  I  that  {Cn> 

is  a  non -arithmetic  renewal  process  whose  intervals  have  finite 

expectation.  From  (2.11)  we  are  able  to  conclude  that  the  sigma 

algebras  a(L(t) .t^C^)  and  o(L(t) ,t>C^}  are  independent  for  each 

k.  Lastly,  every  Sr  L  has  the  same  probability  distribution. 

Lk 

In  other  words,  L  is  properly  regenerative  with  respect  to  (Cn>. 
Remark:  More  generally,  of  course,  Y  is  regenerative  with  respect 
to  (Cn>. 

The  result  pertinent  to  Theorem  1.3  is,  unfortunately  less 
complete : 

Theorem  2.4:  (Wn>  is  regenerative  with  respect  to  the  renewal 
sequence  (N^) . 

Proof:  Since  N,  *  N(C.)  (see  (2.9)),  N.  is  measurable,  and 


-  N.  is/k 


k+m 


measurable;  hence,  the  intervals  (N^+1  -  N^) 


-13- 


k 


are  mutually  independent  according  to  (2.11).  That  they  are 
also  identically  distributed  is  a  consequence  of  (2.12).  More¬ 
over,  has  an  honest  distribution,  or  equivalently,  P[N^  <  »]  »  1. 
To  prove  this,  return  to  the  discrete  parameter  process  {Zn}, 
letting  c^  be  the  k'th  hitting  epoch  for  state  0.  Since  0  is 
positive  recurrent  for  Z,  0  is  likwise  positive  recurrent  for  {Zn>. 
Then  c^  is  almost  surely  finite.  If  we  now  rewrite  (2.9)  in  the 
form 


X 

t»i  *  ^  tH  znn  -ii  zn'1ni* 


(2.13) 


n=l 


and  note  that  each  summand  must  be  finite,  we  have  P[N^  <  «]  *  1. 

It  remains  to  show  that  W^,W2,..  have  the  same  multivariate 

distribution  as  W1+N  ,W2+N  ,..  for  every  k.  Since  Sc  IQ  n  *  Ik  n 

k  k  k  *  * 

it  follows  from  (2.12)  that  the  joint  distribution  of  the  Iq  n 

coincides  with  that  of  the  I.  „ .  The  definition  (2.8)  of  W  ... 

K,n  v  n+N^ 

then  leads  to  the  desired  result. 


Corollary  2.5:  Almost  surely 


N, 


n*l 


W  <  00 
n 


(2.14) 


Proof:  For  each  n  *  1,2,..,N,  ,  we  have  W  <  C. .  Since 

l  n  i 

P[C^  <  *]  *  1  from  E(C^)  <  ®,  the  proof  is  complete. 

Although  both  sides  of  (1.1)  are  seen  to  be  finite  from  the 
corollary  just  stated,  more  is  needed.  None  of  our  assumptions 
preclude  any  of 


E(N) 


N  rC 

,  E[^  Wn]  -  «  ,  E  [  /  L(t)dt] 

n-1  J0 


- - -  Ii  !  .1  a  I,  I  in'.  I  '  -•  **  ~'.‘l 


(2.15)  I 


I 


in  which  we  have  again  written  N  for  and  C  for  C^.  The  first 
expression  in  (2.15)  is  contrary  to  a  hypothesis  in  Theorem  1.3, 
in  which  finiteness  is  needed  to  make  sense  of  (1.3)  and  (l.S). 

The  second  and  third  expectations  in  (2.15)  are  finite  or  infinite 
together,  as  may  be  seen  from  Theorem  2.1.  To  illustrate  (2.15), 
we  offer  an  example  involving  a  single  Markov  server. 

Example  2.6:  From  (2.2),  the  single  Markov  server  is  described 
by  {q^}  and  {a(i)>.  We  take  3  1  and  with  0  <  k^  <  k2  <  .. 

let  qk  k  *  p'  qk  0  *  1_p-  Then 
n  n+1  n 

00 

E(C)  -  p'1  ^  EaCVJ'V  ’  (2.16) 

n*l 

so  that  Z  is  recurrent  for  any  p  <  1  provided  that  lim[n/ct(k  )  ]  >_  1 

n  n 

this  is  an  easy  consequence  of  the  theory  of  power  series  ([10], 
Section  5.4).  We  further  find  that 

00 

E(N)  -  (l-p)p'1  Y]  k  pn  .  (2.17) 

*  n 

n*l 

It  is  apparent  that  the  free  choice  of  the  rate  of  growth  of  (kn> 

allows  on  the  one  hand  the  possibility  that  E(N)  <  00  for  any 

p  <  1,  or  on  the  other  (by  taking  T!m[nvTC]  *  *)  that  E(N)  «  « 

n  n 

for  every  p  >  0.  Finally,  we  calculate 

/C  *  • 

i(t)dt)  -  p'1  ^2  *  C2’18) 

u  n-1 

Both  finite  and  infinite  expectations  in  (2.17)  and  (2.18)  are 
compatible  with  E(C)  <  *  in  any  combination,  as  we  shall  show. 


(a)  Let  {k  }  satisfy  1  <  lim  [n,/Jc“]  <  Tim  [n4c“]  -  p'1<  ®, 
n  —  — — —  n  —  _  n  o 

n  n 

and  take  a(kn)  ■  kn*  Then  E(C)  <  <*>  and 


E[  I  L(t)dt]  <  » 


(2.19) 


for  any  p  <  1,  while  E(N)  ■  «  for  p  >  pQ  and  E(N)  <  ® 
for  p  <  pQ. 

(b)  If  o(kn)  *  1  for  all  n,  E(C)  <  «  for  any  p  <  1.  Here 
(2.10)  is  true  iff  E(N)  <  «. 

(c)  We  choose  {kn >  and  {afk^)}  such  that  E(C)  <  «,  E(N)  <  ®, 


E (  /  L(t)dt]  -  ® 


(2.20) 


Take  lim  ["/onCT]  «  l,  and  Tim  [n/lT]  *  2;  the  radii 
n  n  5  n  n 

of  covergence  of  the  power  series  (2.16),  (2.17)  and 

2  1  1 

(2.18)  are  then  respectively  j,  j,  and  j,  leading  to 
different  possibilities  that  depend  on  the  selection  of 
p.  Specifically,  for  j  <  P  <  j  we  obtain  the  asserted 
expectations . 

One  might  infer  from  the  example  that  some  of  the  anomalies  in 
infinite  expectation  are  connected  with  the  possibility  of  customer 
entry  in  arbitrarily  large  batches.  Accordingly,  we  study  the 
behavior  of  these  expectations  under  constraints  on  the  number 
of  customers  that  can  enter  at  any  one  time. 

Theorem  2.7:  Suppose  q ■  0  whenever  z  ■  x+ke,  for  k  >  k_. 

.  aZ  X  0 

where  kQ  is  a  fixed  integer.  Then  E(N)  <  ®. 


aMNIMMMfilflttMj 


-16- 


Proof:  From  (2.13),  <  kQc^  and,  since  the  positive  recurrence 

of  Z  implies  that  of  {Zn},  E(c^)  <  ®. 

Remark:  There  is  no  point  in  analyzing  systems  with  E(C)  ■  », 

since  then  L  and  W  are  both  infinite. 

It  is  tempting  to  conclude  from  Theorem  2.7  that  bounded 

rC 

inputs  yield  also  the  finiteness  of  E [  /  L(t)dt].  Unfortunately, 

y0 

such  an  assertion  is  false  in  the  absence  of  additional  hypotheses, 
so  that  the  finiteness  of  L  and  W  cannot  be  taken  for  granted. 

In  the  following  illustration,  we  see  that  (2.20)  can  apply  even 
for  the  single  server  Markov  queue  with  individual  arrivals  and 
services. 

Example  2.8:  The  discrete  parameter  embedded  Markov  chain  is  a 
simple  random  walk  (birth-and-death  process)  with  qQ1  »1,  q1(j 

<12  ■  T>  and  for  "  i  2>  Vn*l  •  and  Vn-1  ‘  7^-  lf 

we  take  a(n)  *  1  for  all  n,  it  follows  that  the  relevant  expecta¬ 
tions  can  be  written  in  terms  of  the  discrete  parameter  process 


1 

7’ 


(Zn),  that  is,  E(C) 


E(c)  and 
■C 


I 


E[  L(t)dt] 


E[^Zn] 

n-1 


(2.21) 


Following  [4],  Section  1.12  and  1.14,  one  calculates  (ignoring 
periodicity)  that  E(c)  <  »,  and  that  for  n  >_  2  the  stationary 
probabilities  satisfy  vn  ■  2 [(n+1) (n-1) ] "1vQ.  Moreover, 

c  ■ 

E[JV]  -  v;^nvn  -  -  (2.22) 

n-1  n-1 


The  above  result  remains  qualitatively  unchanged  for  other  (a(n)} 
provided  only  that  0  <  inf  (a(n)]  and  sup  [o(n)]  <  «.  Thus,  there 
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appear  to  be  reasonable  Markov  queueing  models  for  which  HCL^) 
and  ECW^)  are  both  infinite. 

In  view  of  Theorems  2.4  and  2.7,  Theorem  1.3  is  valid  under 
the  additonal  assumption  that  N  is  aperiodic.  Nevertheless, 
unless  L  and  W  are  finite,  we  cannot  arrive  at  Little's  formula. 
We  must  therefore  seek  conditions  under  which 


i: 


E [  |  L(t)dt]  <  « 
'0 


(2.23) 


We  now- -and  hereafter- -suppose  that  only  one  customer  can  enter 
at  a  time  (i.e.,  kQ  *  0) .  We  also  assume  (and  shall  shortly  discuss 
in  detail)  that  the  interarrival  times  mentioned  in  Theorem  1.4 
are  indeed  i.i.d.  random  variables.  Then  we  shall  have 
Theorem  2.9:  Let  E(N2)  <  ®.  Then  (2.23)  holds. 

fC 

Proof:  We  will  majorize  the  integral  /  L(t)dt  as  follows. 

0 

Let  V.  -  inf {t :M(t)*j } ,  so  that  A.  *  V. -V.  ..  Since  C  <  V^+1 

J  J  J  J 

and  L(t)  <  j  on  (Vj>Vj+j]  we  obtain 

C  N  N+l  » 

L(t)dt  <  ^jAj+i  <  "  S^AjI{j<N+l}  »  (2.24) 

0  j-1  j-1  j-1 

in  which  I  is  a  self-evident  indicator  function.  On  the  right 

of  (2.24),  the  event  {N£j-2>  must  be  independent  of  A^  (an 

interarrival  time  belonging  to  the  next  busy  cycle).  Consequently, 

the  expectation  of  the  right  side  of  (2.24)  simplifies;  with 
-1 


f 

»  A 


E(Aj)  ■  X  ,  we  have 


. 


E[  I  L(t)dt]  <  X"1[E(N2)+E(N)] 


(2.25) 
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2 

Remark:  It  may  be  inconvenient  to  verify  whether  E(N  )  is  finite. 

However,  N  <  c,  and  the  recurrence  time  c  is  completely  described 

in  terms  of  the  transition  matrix  Q  for  {Zn>;  see  [4],  Section  I. 11 

2 

for  specific  properties  of  E(c  ).  We  also  note  that  there  is  no 
need  to  assume  that  the  Aj  have  finite  expectation.  In  fact, 

(and  hence  every  A^)  is  exponentially  distributed  with  parameter 
a(0)  . 

So  far,  we  have  studied  the  applicability  of  Theorems  1.1 
through  1.3  for  Markov  queueing  systems.  Now  we  turn  to  the 
substance  of  Theorem  1.4.  As  we  have  noted  above,  A^  has  an 
exponential  distribution  with  parameter  a(8) ,  so  that  {A^}  is 

^  •  t  «t*V 

a  Poisson  process  if  the  variates  are  i.i.d.  in  accordance  with 
the  hypotheses  of  Theorem  1.4.  It  is  essential  to  realize  that 
the  interarrival  intervals  A^  are  wholly  expressed  within  the 
structure  of  Z;  the  arrival  process  M  is  incremented  whenever 
the  state  z  jumps  to  z+e^  for  some  i  *  l,2,..,r.  However,  even 
for  a  Poisson  process  K  induced  by  a  Markov  Z,  intervals  such  as 
those  described  in  Theorem  1.4  need  not  be  i.i.d.  when  the  indices 
are  stopping  times  for  instead  of  the  smaller  sigma  algebra 
■  a{M(u),u<t}.  A  simple  example  illustrates  this  phenomenon. 
Let  {D^}  be  the  interdeparture  intervals  associated  with  the 
departure  process  K  of  a  M/M/1  queue  in  equilibrium,  and  take  C 
as  the  time  when  the  system  first  becomes  empty.  It  is  known  that 
K  is  Poisson;  nevertheless,  *  inf  {t :K(t) -K(C)»1>  -  C  is 

neither  exponentially  distributed,  nor  independent  of 
Thus,  we  must  assure  that  the  A^  are  i.i.d.  by  an  assumption  that 
goes  beyond  the  independence  of  (M(t)-M(s)]  from  to  permit 
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the  application  of  {$£}  stopping  times.  Specifically,  the  evolu¬ 
tion  of  the  arrival  process  must  be  independent  of  the  current 
state  of  the  queueing  system,  as  is  actually  true  whenever  the 
customer  stream  is  not  influenced  by  internal  system  conditions. 

We  formalize  the  above  discussion  through 

Hypothesis  2.10:  lim  h_1P(M(s+h) -M(s) =1 | Z (s) ]  =  X  (2.26) 

h\0 

for  all  s  ^  0. 

Evidently,  Hypothesis  2.10  states  that  the  rate  of  accession 
of  customers  into  the  system  depends  neither  on  the  time  nor 
on  the  state  of  the  system.  The  Hypothesis  provides  some  of  the 
prerequisites  for  Theorem  1.4,  as  is  indicated  by 
Theorem  2.11:  Hypothesis  2.10  is  equivalent  to 

P[M(t)-M(s)«n|^c  ]  »  CA.(y)]n  e-X(t-s)  (2.27) 

for  every  n  and  t  ^  s  >  0.  If  one  of  these  conditions  is  satisfied, 
M  is  a  Poisson  process,  and  the  are  i.i.d.  random  variables.1 

Proof:  In  the  presence  of  (2.26),  the  forward  Kolmogorov  equation 
for  the  Markov  process  Z  becomes 

^■{P[M(t)-M(s)-n|Z(s)»z]}  -  X{P[M(t)-M(s)«n-l|Z(s)»z] 

( 2 . 28) 

-  P[M(t) -M(s)»n j Z(s)*z]  } 

where  (M(t) -M(s)«-1}  has  zero  probability,  and  an  initial 

1If  the  conditioning  in  (2.26)  is  on  c  M  is  already  a  Poisson 
process  respective  to  } ,  without  reference  to  the  Markov  nature 
of  Z.  One  notes  that  the  compensator  of  the  submartingale  M  is 
given  by  [IS],  VII. T28  and  VII. T29,  which  is  almost  surely  equal 
to  Xt  by  comparison  with  (2.26).  The  desired  result  then  follows 
from  Watanabe's  theorem  ([5],  p.  76). 
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condition  is  furnished  by  P[M(s) -M(s)=0]  -  1.  If  we  now  solve 
the  system  (2.28)  recursively,  we  obtain  (2.27),  except  that  the 
conditioning  is  on  Z(s).  However  Z  is  Markov,  so  that  the 
conditioning  on  Z(s)  may  be  replaced  by  the  sigma  algebra  &'s- 

To  show  that  M  is  Poisson,  take  the  conditional  expectation 
of  (2.27)  with  respect  to *  a{M(u),U£S}.  Since 
(2.27)  remains  true  if  the  conditioning  is  onJif  rather  than 
with  respect  to  «^.  Next,  we  consider  the  A^.  As  before, 
we  define  Vk  *  inf  {t :M(t)=k}.  Then  a{A^  ,j_<k}Ce^  ,  and  we  show 
that  Ak+^  is  exponentially  distributed  and  independent  of  . 

In  fact, 

P[Ak+1>a|^v  ]  -  P[M(Vk+a)-M(Vk)  =  0|^  ]  ;  (2.29) 

k  lc 


by  the  strong  Markov  property,  and  because  of  (2.28),  the  right 
side  of  (2.29)  is  simply  e"*a.  This  demonstrates  both  the 
independence  and  exponential  distribution  of  the  Ak> 

Lastly,  (2.27)  is  readily  derived  from  (2.28),  so  the 
proof  of  the  Theorem  is  complete. 

Corollary  2.12:  X  *  a(0)  (2.30) 

Proof:  In  (2.27),  let  n  ■  0  and  s  *  0.  Since  {M(t)=0}  *  {T^>t}, 
we  can  compare  (2.27)  with  P(T^>t)  as  obtained  from  (2.2). 

We  could  now  show  the  applicability  of  Theorem  1.4  when 
Hypothesis  2.10  holds.  Although  the  length  of  the  first  busy 
cycle,  as  defined  herein,  does  not  satisfy  (1.4),  one  can  show  that 

N+l 

E(C)  -  E[^Ak]  -  E(AX)  ,  (2.31) 

1 


from  which  (1.5)  eventually  follows.  However,  Hypothesis  2.10 


permits  us  to  proceed  more  directly  via  a  simpler  proof,  which 
moreover  requires  no  recourse  to  Wald's  Lemma. 

Theorem  2.13:  Under  Hypothesis  2.10,  we  have 


E (C)  -  X*  XE (N)  .  (2.32) 

Proof:  The  process 

K(t)  -  M(t)  -  Xt  (2.33) 

is  a  local  martingale  of  zero  mean  under  the  family  of  sigma 
algebras  {^.};  this  is  seen  by  computing  E [M(t)  -M(s)  | from 
(2.27).  According  to  the  optional  sampling  theorem  ([IS],  Theorem 
VI.T13),  the  martingale  relation  remains  valid  if  ordinary  times 
are  replaced  by  stopping  times  respective  to  C  is  such  a 

stopping  time,  and  M(C)  =  N.  Hence,  the  evaluation  of  E[K(C)]  *  0 
yields  (2.32)  directly. 

The  results  obtained  thus  far  can  be  summarized  to  enunciate 
sufficient  conditions  under  which  Little's  formula  is  valid  for 
a  Markov  queueing  system. 

Theorem  2.14:  Let  Z  be  an  irreducible  positive  recurrent  Markov 
queueing  process.  Assume  that  customers  enter  the  system  singly, 
and  that  the  entry  process  satisfies  Hypothesis  2.10.  Suppose 
further  that  the  number  N  of  customers  in  a  (any)  busy  cycle  is 
aperiodic,  with  E(N  )  <  *.  Then  L(t)  converges  in  distribution 
to  a  random  variable  whose  expectation  L  is  finite,  where  L(t)  is 
the  number  of  customers  in  the  system  at  time  t.  Similarly,  the 
sojourn  times  Wn  of  customers  (ordered  by  their  chronology  of 
entry)  converges  in  distribution  to  a  variable  of  finite  expecta¬ 
tion  W.  These  expectations  are  related  through  Little's  formula, 

L  -  XW  ,  (2.34) 


in  which  X  is  the  expected  rate  of  entry  of  customers. 

III.  TWO  EXTENSIONS  OF  LITTLE'S  FORMULA 

In  this  Section  we  examine  the  effects  of  some  relaxations 
of  the  hypotheses  required  for  Theorem  2.14.  The  appropriate 
questions  are:  does  Little's  formula  still  hold,  perhaps  in 
modified  form?  If  so,  how  must  (2.34)  be  changed?  The  most 
likely  candidates  for  less  restrictive  assumptions  are  those 
on  the  periodicity  of  N,  and  the  state  invariance  (see  Hypothesis 
2.10)  of  the  customer  entry  rate.  Each  of  these  is  analyzed  in 
turn. 

We  have  already  seen  from  Example  0.1  that  N  may  be  periodic; 
in  fact,  the  periodicity  of  N  is  trivial  if  we  permit  only  batch 
service  of  a  single  specified  size.  However,  N  can  also  be 
periodic  in  simple  inifinite  capacity  systems  with  unit  arrivals 
and  service,  as  is  indicated  by 

Example  3.1:  The  periodicity  of  N  depends  only  on  the  set  of 
possible  transitions  of  the  discrete  process  {Zn},  so  that  we 
must  specify  which  of  the  q  (see  (2.2))  are  zero.  The  other 

A 

qxz  and  the  a(z)  are  arbitrary,  subject  to  *  1  and  the 

z 

positive  recurrence  of  Z.  With  this  in  mind,  we  concentrate  on 
specifying  allowable  one-step  transitions  of  {Zn}.  Specifically, 
we  construct  these  transitions  to  give  N  a  periodicity  of  two, 
although  the  same  technique  can  be  used  to  yield  other  periodi¬ 
cities  for  N. 

The  state  space  X  of  Z  is  partitioned  into  subsets 
*  {z:|{z||ak>.  Let  J(0)  »  0,  and  each  X^,  k  ^  1,  is  the  union  of 
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disjoint  nonempty  subsets  I (k)  and  J(k).  The  following  one-step 
transitions  are  the  only  ones  allowed:  J(0)->-I(l),  and  for  each 
k  >  1,  I  (k)-*J(k+l)  ,  J(k)-*-I(k+l)  ,  and  J(kW(k-l) .  If  Mn  is  the 
number  of  customer  arrivals  in  the  first  n  epochs,  M11  is  even 
if  ZneJ(k)  and  odd  if  Znel(k)  (any  k) ;  this  is  readily  shown  by 
induction.  Therefore,  M11  is  even  whenever  Zn  *  0,  and  this  reflects 
the  periodicity  of  N. 

For  instance,  we  may  take  a  system  consisting  of  two  dis¬ 
connected  exponential  servers  with  state -dependent  assignment 
of  incoming  customers,  and  service  by  only  one  server  at  each 
state.  In  terms  of  the  I (k)  and  J(k),  we  let  J(2m)  *  (m,m) , 

J(2m+1)  *  (m,m+l) ,  I(2m)  a  (m-l,m+l),  and  I(2m+1)  »  (m+l,m). 

As  was  noted  earlier,  service  and  customer  accession  rates  may 
be  chosen  arbitrarily. 

The  effect  of  periodic  M  on  Little's  formula  is  minor,  as 
is  indicated  by 

Theorem  3.2:  Let  the  number  of  customers  in  a  busy  cycle  have 
periodicity  d.  Then  the  conclusions  of  Theorem  2.14  remain 
unchanged,  except  that  the  sojourn  times  Wn  may  fail  to  converge 
in  distribution.  Instead,  as  n  -*■  • 

d 

d'1Z  wnd.j  *  W-  -  t3-1] 

j-l 

the  convergence  again  being  in  distribution. 

Proof:  Theorem  3.2  in  [6]  is  modified  by  insertion  of  the  dis¬ 
crete  renewal  theory  result  ([5],  p.  313)  for  periodic  renewal 
processes.  In  other  words,  the  basic  formula  L  *  XW  remains 
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unchanged,  with  the  meaning  of  W,,,,  generalized  by  (3.1)  to  arbitrary 
d  >  1. 

We  next  turn  to  the  consideration  of  arbitrary  customer  entry 
rates,  which  are  assumed  to  depend  on  the  state  of  the  system,  but 
not  directly  on  time.  We  shall  suppose  (as  before)  that  customers 
arrive  singly,  but  that  (cf.  (2.26)) 

lim  h'1P[M(s+h)-M(s)-l |Z(s)»z]  =  X,  (3.2) 

hXO  2 

need  not  be  a  constant.  Call 

/v 

X  *  sup  X  ,  (3.3) 

zeX  2 

A 

and  assume  that  X  is  finite.  We  can  then  modify  the  system  as 

follows:  whenever  Z(t)  «  z,  there  is  a  additional  Poisson  customer 
^  A 

input  stream  M  of  intensity  (X-X_),  which  is  conditionally  independ- 

it 

ent  of  M,  given  Z(t).  Any  arrival  under  M  passes  through  the  system 
instantaneously,  so  that  Z  is  unchanged,  and  the  corresponding 
Wn  -  0.  The  total  arrival  stream  (M+M)  then  satisfies  Hypothesis 
2.10.  To  investigate  the  other  hypotheses  underlying  Theorem  2.14, 
we  first  note  that  a  rigorous  construction  of  the  modified  process 
Y  including  the  "passed  through"  customers  is  easily  achievable. 

One  augments  Y  by  M,  and  adjusts  U  to  properly  index  the  arriving 
customers  to  include  those  attributable  to  M.  We  shall  omit  the 
details,  which  are  tedious  and  routine.  Let  us  denote  the  para¬ 
meters  of  the  modified  process  consistently  by  carats.  Relations 
between  the  old  and  new  processes  include 

[  L(t)dt  -  f  L(t)dt  (3.4) 

J  0  •'O 
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and 

N  N 

XX  '  XX  (3-S> 

n**l  n*l 

2 

If  E(N  )  <  «*,  Theorem  2.9  assures  that  all  the  expectations  of  the 
terms  in  (3.4)  and  (3.5)  are  finite.  Moreover,  N  is  aperiodic 

if  X  <  X  for  at  least  one  z.  Theorem  2.14  then  asserts  that 

2 

£  =  XW  .  (3.6) 

Since  L(t)  *  L(t)  for  each  t,  L  can  be  replaced  by  L  in  (3.6). 

A  formula  involving  L  and  W  can  then  be  derived  if  W  is  related  to 

A 

w. 

Indeed,  (1.3)  is  valid  for  both  W  and  W,  so  that 

W  -  [E(N)]{E(N)>'1W  (3.7) 

in  view  of  (3.5).  To  evaluate  the  ratio  E(N)/E(N),  take  N  as  the 
number  of  customers  entering  the  original  system  during  the  busy 
cycle  while  Z(t)  *  z.  Now 

E[N  -N  |Z(t)»z  for  duration  x_]  *  (X-X_)t_  (3.8) 

2  2  2  2  2 

and  (see  [5],  p.  263) 

E(tz)  «  *ZE(C)  ,  (3.9) 

in  which  it  is  the  equilibrium  probability  t  «  lim  P[Z(t)«z]. 

Z  2  t^-09 

The  resulting  E(N  -N  )  •  E(C)  <(A-X_)tt_}  can  be  summed  to  obtain 

2  2  2  2 

E(N-N).  Moreover,  Theorem  2.13  applies  to  the  modified  system, 
so  that  E(£)  ■  x’1E(N),  with  E(6)  ■  E(C)  because  2  »  C.  It 
follows  that 

E(N)  -  E(C){£x2Trz} 
zeX 


(3.10) 


which  is  itself  a  useful  formula,  and 

ecnmecn))'1  -  •  (3.11) 

z 

By  virtue  of  (3.11),  we  can  claim 

Theorem  3.2:  Let  Z  be  an  irreducible  positive  recurrent  Markov 

queueing  process  with  single  customer  entry  at  rates  X  specified 

z 

2 

by  (3.2).  If  E(N  )  <  «,  L(t)  converges  in  distribution  to  a  random 

variable  whose  expectation  L  is  finite,  and  the  Wn  converge  in  the 

sense  of  Theorem  3.1  to  a  random  variable  of  expectation  W.  L 

and  W  are  related  through  Little’s  formula,  which  takes  the  form 

L  *  XW  .  (3.12) 

Here  X  is  defined  as  X  *  (VX  ir  ) . 

z  2  2 

Evidently,  X  can  be  identified  with  the  mean  customer  entrance 
rate,  since  ir z  is  also  the  average  proportion  of  time  the  system 
spends  in  state  z.  Indeed,  X  has  the  further  interpretation  as 
the  almost  sure  limit  of  t"*M(t) ,  so  that  X  appears  when  Little's 
formula  is  taken  in  the  time  average  sense  (cf. [18]). 

VI.  PRIORITY  INTERRUPTS  AND  CLASSES  OF  CUSTOMERS 

To  broaden  the  applicability  of  Little's  relation  further, 
we  extend  our  model  to  admit  network  behaviors  discussed  in  recent 
(computer  oriented)  papers  [1][3][13].  We  shall  begin  with  a 
consideration  of  priority  interrupt  queueing  disciplines,  in  which 
a  customer  currently  in  service  is  pushed  aside  in  favor  of  a  new 
arrival;  once  the  new  customer  completes  service  at  the  station, 
the  order  of  resuming  incompleted  services  is  prescribed  by  the 
queueing  discipline. 


* ■ . .... 
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Because  of  the  "forgetting”  property  of  the  exponential 
distribution,  an  interrupt  at  an  exponential  server  is  equivalent 
to  a  mere  reordering  of  the  customers  at  the  station.  This  is 
easily  provided  by  the  specification  of  {Un}.  For  a  service 
distribution  whose  Laplace  transform  is  rational,  the  approach 
is  more  complex.  Ordinarily,  the  single  station  is  represented 
as  m  substations,  where  m  is  the  degree  of  the  denominator  of  the 
rational  transform  [7]  [17].  Appropriate  blocking  (no  service 
provided  at  more  than  one  substation  at  a  time)  then  leads  to  the 
correct  model.  This  structure  can  be  extended  to  priority  interrupts 
by  paralleling  all  but  the  first  substation  by  "dummy  substations" 
incapable  of  providing  service.  Upon  arrival  of  a  new  customer, 
the  customer  (if  any)  at  substation  k  is  "laid  aside"  to  the 
corresponding  dummy  station.  When  the  new  customer  has  passed 
through  all  m  substations,  one  of  the  customers  who  has  been 
laid  aside  is  selected  by  the  discipline  to  continue  service  by 
returning  immediately  to  the  real  substation  from  whence  he  came. 
Since  all  substations  function  as  exponential  servers,  the  above 
technique  becomes  valid  by  virtue  of  the  "forgetting"  property 
at  each  substation. 

The  construction  of  the  preceding  paragraph  leaves  untouched 
the  consistency  of  (Un }  and  (Zn),  so  that  Theorem  2.1  holds  as 
before.  Thus  (cf.  Theorem  2.14),  Little's  formula  holds  for 
priority  interrupt  systems  under  the  same  hypotheses  as  for  any 
of  the  other  disciplines  mentioned.  Naturally,  the  extensions  of 
Section  III  will  apply  to  priority  interrupt  queues  also. 

We  now  turn  to  systems  with  multiple  customer  types.  Several 


versions  of  Markov  queueing  systems  with  different  classes  of 


customers  have  been  proposed  [1]  [13] ,  but  not  all  of  these  are 
consistent  with  Little's  result  for  the  individual  classes.  In 
particular,  transmutation  of  customers  from  one  class  to  another 
can  invalidate  Little's  formula.  A  simple  example  is  adduced  by 
considering  tandem  exponential  servers,  with  entrance  of  class 
one  customers  only  to  the  first  server.  As  each  customer  passes 
on  to  the  second  server,  he  becomes  a  class  two  customer,  and 
eventually  departs  as  such.  A  direct  calculation  shows  that 
Little's  equality  holds  for  neither  customer  class.  To  avoid  such 
difficulties,  our  model  maintains  the  integrity  of  the  class  of 
each  customer  as  he  passes  through  the  network.  However,  we  do 
not  restrict  priority  assignments,  or  service  rates  and  routings 
depending  on  the  number  and  types  of  customers  at  each  service 
station. 

A  prefix  is  used  to  denote  each  of  the  c  customer  classes. 
Specifically,  the  line  length  process  at  station  j  takes  on  the 
vector  form  Z.  »  Z ^ ,2Zj , . ,QZ. ) .  Similarly,  is  the  rate 

of  arrival  for  customers  of  class  i,  is  the  corresponding 
arrival  process,  the  waiting  time  for  the  n'th  customer  of 
class  i,  and  so  forth.  It  is  necessary  to  expand  (Un>  so  that 
U11  has  entries  (m,i.a  .q  ,j  ).  In  this  quintuple,  m  is  the 
customer's  order  of  arrival  in  chronological  order,  im  is  his 
class,  am  is  the  customer’s  order  of  arrival  among  customers  of 
class  im,  qm  the  queueing  station  to  which  customer  m  is  assigned, 
and  his  position  in  the  queue  at  qffl.  As  before,  qm  •  jm  *  0  for 
a  customer  who  has  already  exited  the  system. 

We  would  anticipate  that  Little's  formula  appears  as 
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.L  -  (^.W  ;  (4.1) 

such  is  indeed  the  case.  In  the  first  place ,  Theorem  2.1  now 
takes  the  form 


1N  ,C 

Z iWn  '  /  i 
n*l  ■'O 


L(t)dt 


C4.2) 


Next,  as  in  Theorem  2.4,  each  {^W^}  is  regenerative  with  respect 

2 

to  the  renewal  sequence  If  is  aperiodic  and  if  E(N  ), 

we  can  claim 


■  N 


(4.3) 


n=l 


and  (cf.  Theorem  2.9) 


tL  *  (E(C) >”1E{  f  iL(t)dt} 
Jr i 


(4.4) 


To  complete  the  derivation  of  (4.1),  we  must  relate  E(iN)  to 
E (C) .  For  this  purpose,  let  us  suppose  that  Hypothesis  2.10 
holds  separately  for  each  ^M.  The  martingale  decomposition  of 
Theorem  2.13,  applied  to  at  the  stopping  time  C,  yields 


E^N)  -  (.X)E(C)  .  (4.5) 

Finally,  (4.1)  results  from  combining  (4.2)  through  (4.5).  As 
in  the  case  of  one  type  of  customer,  (4.1)  can  be  generalized 
to  include  periodic  and  state -dependent  customer  entry  rates. 


V.  NON -MARKOVIAN  SYSTEMS 

The  Markov  nature  of  the  queueing  process  Z  is  by  no  means 
essential  to  the  derivation  of  Little's  formula;  for  instance, 
the  formula  applies  to  the  stable  G/G/s  queue  [17] ,  which  is 
certainly  not  Markov.  However,  any  assertions  on  Little's  relation 
pertaining  to  queueing  networks  with  arbitrary  service  and  entry 
characteristics  is  necessarily  limited  to  generalities.  We  can 
say  that  if  Z  satisfies  the  regenerative  properties  (2.11)  and 
(2.12),  meets  the  finiteness  conditions  E(N)  <  •  and  (2.23),  and 
has  a  Poisson  process  exogenous  input,  the  standard  arguments  yield 
L  -  XW. 

When  there  is  only  one  mode  of  entrance  to  the  system,  these 
regeneration  points  can  be  taken  as  the  starting  instants  of  the 
busy  period;  this  suggests  a  new  freedom  in  the  choice  of  arrival 
process.  In  fact,  the  length  of  the  (first)  busy  cycle  is  now 
provided  by  (1.4).  That  Z  is  regenerative  respective  to  {C^} 
carries  the  implication  that  the  interarrival  process  {A^}  is 
regenerative  with  respect  to  numbers  of  customers  {N^}  in  the 
respective  busy  cycles.  If,  further,  N  is  aperiodic  and  E(N)  <  », 
the  An  converge  in  distribution  to  a  random  variable  A  whose 
expectation  is 

N 

E(A)  -  {E(N))'1E{^An>  .  (5.1) 

n*l 

We  see  that  (5.1)  implies  (1.5),  but  without  any  requirements 
that  the  AR  be  i.i.d.  random  variables.  Finally,  if  we  take 
X  ■  [E(A)]”*»  we  recover  Little’s  formula  L  •  XW. 
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An  attempt  to  impose  a  more  specific  probability  structure 
on  Z  seems  to  be  less  successful.  For  instance,  we  find  that  a 
semi-Markov  process  Z  is  also  regenerative  respective  to  {C^} 

(cf.  [5],  Section  10.6),  but  are  unable  to  turn  this  observation 
to  advantage  in  generating  a  broader  class  of  queueing  network 
models.  We  are  also  aware  of  the  feasibility  of  approximating 
general  service  time  distributions  by  Erlang  service  in  the  con¬ 
text  of  Markov  systems  (see  [17]  for  a  rigorous  treatment);  although 
this  approach  works  for  some  aspects  of  the  G/G/s  queue  [17],  there 
seems  little  chance  that  the  analysis  can  be  extended  to  the  approxi¬ 
mation  of  L(t)  and  [Wn>  in  queueing  networks.  Consequently,  we 
have  limited  detailed  consideration  to  Markov  queueing  systems, 
which  are  adequate  to  describe  a  rich  variety  of  queueing  system 
behaviors . 
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